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The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )|3 Responsive to communication(s) filed on 11 December 2003 . 
2a)|3 This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) S Claim(s) 7-43 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) M Claim(s) 1-43 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) 13 The drawing(s) filed on 13 August 2001 is/are: a)E3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

1 1) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)DAII b)D Some*c)D None of: 

1. Q Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 

Attachment(s) 

1) ["I Notice of References Cited (PTO-892) 4) CD Interview Summary (PTO-413) Paper No(s). . 

2) CD Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) CH Notice of Informal Patent Application (PTO-152) 

3) CD Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) CH Other: 
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Claim Rejections - 35 USC § 102 



1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis 
for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed in the United 
States before the invention by the applicant for patent or (2) a patent granted on an application for patent by another filed in the 
United States before the invention by the applicant for patent, except that an international application filed under the treaty 
defined in section 351(a) shall have the effects for purposes of this subsection of an application filed in the United States only if 
the international application designated the United States and was published under Article 21(2) of such treaty in the English 



2. Claims 1-43 are rejected under 35 U.S.C. 102(e) as being anticipated by Pub No US 
2001/0000962 issued to Rajan (hereafter Rajan '962). 

Regarding claims 1, 17 and 33, Rajan '962 discloses: 

(a) at least one multimedia information input interface receiving said multimedia information [Fig 1 
and paragraph 0040] 

(b) a computer processor [Fig 1], coupled to said at least one multimedia information input interface 
receiving said multimedia information therefrom, 

processing said multimedia information by performing object extraction processing to generate 
multimedia object descriptions from said multimedia information [scene description information, 
paragraph 0042] 

processing said generated multimedia object descriptions by object hierarchy processing to generate 
multimedia object hierarchy descriptions indicative of an organization of said object descriptions 
[paragraph 0043] 



language. 
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wherein at least one description record including said multimedia object descriptions and said 
multimedia object hierarchy descriptions is generated for content embedded within said multimedia 
information [paragraph 0044] 

( c ) a data storage system, operatively coupled to aid processor for storing at least said at least one 

description record [Fig 1, 105, 176, 186, 160, 170] 

Regarding claims 2, 18 and 34, Rajan '962 discloses image object description [paragraph 0043] 
Regarding claims 3, 7, 19 and 23, Rajan £ 962 discloses image segmentation [composition layer, 

paragraph 0045] and feature extraction [pitch of a sound, paragraph 0045] 

Regarding claims 4, 20, 35 and 39, Rajan '962 discloses size [scale, paragraph 0044] 
Regarding claims 5, 21 and 36, Rajan discloses spatial characteristics [paragraph 0044] 
Regarding claims 6, 22, 37, 41 and 43, Rajan £ 962 discloses logical descriptions [paragraph 

0043] 

Regarding claims 8 and 24, Rajan '962 discloses encoding the image object hierarchy 
descriptions [paragraph 0042] 

Regarding claims 9, 25 and 38, Rajan '962 discloses video object hierarchy descriptions 
[paragraph 0040] 

Regarding claims 10 and 26, Rajan '962 discloses temporal video segmentation [paragraph 
0044] region segmentation [positioning parameters, paragraph 0043], feature extraction [paragraph 
0045] 

Regarding claims 1 1 and 27, Rajan '962 discloses an event feature description of time 
[paragraph 0044] and an object feature description of shape size position motion and time [paragraphs 
0042-0045] 

Regarding claims 12, 28, 40 and 42, Rajan '962 discloses physical event hierarchy descriptions 
[paragraph 0044] and temporal descriptions [paragraph 0044] 
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Regarding claims 13, 14, 29 and 30, Rajan '962 discloses logical event hierarchy organization 
[paragraph 00043], video objects [paragraph 0040] and temporal descriptions [paragraph 0040] 

Regarding claims 15 and 31, Rajan '962 discloses temporal video segmentation [paragraph 
0041] grouping of video events into regions [paragraph 0043] and feature extraction [paragraph 0045] 

Regarding claims 16 and 32, Rajan '962 discloses encoded description information [paragraph 

0042] 



Applicant's arguments filed 12/1 1/2003 have been fully considered but they are not persuasive. 
First Applicant Argument: 

Applicant states on page 13 "Rajan does not disclose or suggest at least several features of the 
claimed invention including 'processing said multimedia information by performing object extraction 
processing to generate multimedia descriptions.' On p 3 of the Office Action paragraph 42 is cited as 
allegedly disclosing this element - however, paragraph 42 is devoid of any discussion of object 
extraction even generally. Indeed, the object extraction of the present invention is a complex process 
defined with reference to another patent application, U.S. Patent Application Serial No. 09/405,555 , 
filed September 24, 1998, entitled 'An Active System and Algorithm for Semantic Video Object 
Segmentation,' the contents of which are incorporated by reference into the specification of the present 
application. Rajan paragraph 42 does not disclose or even remotely suggest this claimed object 
extraction feature in toto or even generally." 
First Examiner Response: 

Examiner is not persuaded. In response to applicant's argument that the references fail to show 
certain features of applicant's invention, it is noted that the features upon which applicant relies (i.e., 
the object extraction of the present invention is a complex process defined with reference to another 



Response to Arguments 
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patent application, U.S. Patent Application Serial No. 09/405,555 , filed September 24, 1998, entitled 
'An Active System and Algorithm for Semantic Video Object Segmentation) are not recited in the 
rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the 
specification are not read into the claims. See In re Van Geuns, 988 F.2d 1 181, 26 USPQ2d 1057 
(Fed. Cir. 1993). 

In response to applicant's argument that the references fail to show certain features of 
applicant's invention, it is noted that the features upon which applicant relies (i.e., Rajan paragraph 42 
does not disclose or even remotely suggest this claimed object extraction feature in toto or even 
generally) are not recited in the rejected claim(s). Although the claims are interpreted in light of the 
specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 
F.2d 1181,26USPQ2d 1057 (Fed. Cir. 1993). 

Furthermore, examiner maintains that Rajan in paragraph 43 does in fact disclose the claim 1 
limitation "processing said multimedia information by performing object extraction processing to 
generate multimedia object descriptions from said multimedia information, and processing said 
generated multimedia object descriptions by object hierarchy processing to generate multimedia object 
hierarchy descriptions indicative of an organization of said object descriptions." 

Examiner considers below the specification on page 12 and continuing onto page 13 where the 
following is recorded: 

The third subsystem 230 is a video object segmentation system in which automatic 
segmentation is integrated with user input to track semantic objects in video sequences. For 
general video sources, the system allows users to define an approximate boundary by using a 
tracing interface. Given the approximate object boundary, the system automatically refines 
the boundary and tracks the movement of the object in subsequent frames of the video. The 
system is robust enough to handle any real-world situations that are hard to model in 
existing approaches, including complex objects, fast and intermittent motion, complicated 
backgrounds, multiple moving objects and partial occlusion, the description generated by 
this system is a set of semantic objects with the associated regions and features that can be 
manually annotated with text. A complete description of the system 230 is contained in U.S. 
Patent Application Serial No. 09/405,555, files September 24,1998, entitled "An Active 
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System and Algorithm for Semantic Video Object Segmentation," the contents of which are 
incorporated by reference herein. 

Examiner maintains that the disclosure by Rajan in paragraphs 43-46 reads on the above claim 
language and disclosure by Rajan: 

43. The scene description information describes the logical structure of a scene, and 
indicates how objects are grouped together. Specifically, an MPEG-4 scene follows a 
hierarchical structure, which can be represented as a directed acyclic (tree) graph, where 
each node or a group of nodes, of the graph, represents a media object. The tree structure is 
not necessarily static, since node attributes (e.g., positioning parameters) can be changed 
while nodes can be added, replaced, or removed. 

44. The scene description information can also indicate how objects are positioned in space 
and time. In the MPEG-4 model, objects have both spatial and temporal characteristics. 
Each object has a local coordinate system in which the object has a fixed spatial-temporal 
location and scale. Objects are positioned in a scene by specifying a coordinate 
transformation from the object's local coordinate system into a global coordinate system 
defined by one more parent scene description nodes in the tree. 

45. The scene description information can also indicate attribute value selection. Individual 
media objects and scene description nodes expose a set of parameters to a composition layer 
through which part of their behavior can be controlled. Examples include the pitch of a 
sound, the color for a synthetic object, activation or deactivation of enhancement 
information for scaleable coding, and so forth. 

46. The scene description information can also indicate other transforms on media objects. 
The scene description structure and node semantics are heavily influenced by VRML, 
including its event model. This provides MPEG-4 with an extensive set of scene 
construction operators, including graphics primitives that can be used to construct 
sophisticated scenes. 



Second Applicant Argument: 

Applicant states on page 13: 
Likewise, paragraph 0043 is cited on p. 3 of the Office Action as allegedly disclosing the claimed 
feature of "processing said generated multimedia object descriptions by object hierarchy processing to 
generate multimedia object hierarchy." However, while paragraph 0043 generally discloses that an 
MPEG-4 scene follows a hierarchical structure, it nowhere indicates that multimedia objection 
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descriptions, which are generated by, e.g., performing object extraction, are processed by object 



Second Examiner Response: 

Examiner is not persuaded. Applicant is referred to above response. 

Third Applicant Argument: 

Applicant states on page 13/14 the following: 
Claims 3, 7, 19 and 23 are patentable for the additional reason that they include limitations of "image 
segmentation" and "feature extraction." On p. 3 of the Office Action, it is asserted that paragraph 0045 
describe these features. In fact, it is asserted in the Office Action that the limitation of "feature 
extraction" is disclosed by the reference to "pitch of a sound" in T 0045. This is plainly incorrect. As 
noted above, the object 

extraction of the present invention is defined with reference to another patent application, U.S. Patent 
Application Serial No. 09/405,555, filed September 24, 1998, entitled "An Active System and 
Algorithm for Semantic Video Object Segmentation," the contents of which are incorporated into the 
specification of the present application. This claimed feature cannot possibly be disclosed or suggested 
by simple reference in Rajan to "pitch of a sound." See Office Action, p. 3. 
Third Examiner Response: 

Examiner is not persuaded. MPEP § 21 1 1.01 requires that tc [d]uring examination, the claims 
must be interpreted as broadly as their terms reasonably allow. This means that the words of the claim 
must be given their plain meaning unless applicant has provided a clear definition in the specification. 
In re Zletz, 893 F.2d 319, 321, 13 USPQ2d 1320, 1322 (Fed Cir. 1989). One must bear in mind that, 
especially in nonchemical cases, the words in a claim are generally not limited in their meaning by 
what is shown or disclosed in the specification. It is only when the specification provides definitions 



hierarchy processing. 
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for terms appearing in the claims that the specification can be used in interpreting claim language. In 
re Vogel, All F.2d 438, 441, 164 USPQ 619, 622 (CCPA 1970) " 
Fourth Applicant Argument: 

Independent claim 33 as amended is directed to a computer readable medium with at least one 
multimedia description record describing multimedia content for corresponding multimedia 
information, the description record comprising, inter alia, object descriptions, generated by performing 
object extraction processing. As discussed at length above, Rajan falls to disclose or even remotely 
suggest at least the feature of object descriptions generated by performing object extraction processing. 
Accordingly, because Rajan falls to disclose or suggest at least these claimed features, this reference 
fails to anticipate independent claim 33 as amended. 
Fourth Examiner Response: 

Examiner is not persuaded. Applicant is referred to above response 1 . 



THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as 
set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE MONTHS 
from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the 
mailing date of this final action and the advisory action is not mailed until after the end of the THREE- 
MONTH shortened statutory period, then the shortened statutory period will expire on the date the 
advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from 
the mailing date of the advisory action. In no event, however, will the statutory period for reply expire 
later than SIX MONTHS from the mailing date of this final action. 



Conclusion 
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Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Etienne LeRoux whose telephone number is (703) 305-0620. The examiner can 
normally be reached on Monday - Friday from 8:00 AM to 4:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Safet Metjahic, can be reached on (703) 308-1436. 

Any inquiry of a general nature or relating to the status of this application or proceeding should 
be directed to the receptionist whose telephone number is (703) 305-3900. 
Etienne LeRoux 
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SUPERVISORY PATENT EXAMINER 
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